The exemplary embodiment relates to the information seeking field. It finds particular application in connection with serendipitous browsing combined with query-based searching in multimedia collections and will be described with particular reference thereto.
Information retrieval systems provide a user-friendly interface by which a user can retrieve documents from a database that are relevant to or match a query. Typically, an information retrieval system ranks a “top N” documents that best match the query. An example of such a system is an Internet search engine.
Many information retrieval systems are text-based. That is, the information retrieval system receives a textual query and searches textual content of documents for similarities with the textual query, such as the same or similar words or terms, common semantic content (based, for example, on derivation of semantically related words determined using an on-line thesaurus), and the like. In a more complex approach, language models may be developed to represent the query and documents to be searched, and the information retrieval is based on similarity of query and document language models.
Digital information repositories enable storage and processing of information in different media types or “modalities,” such as text, images (single images or video), audio, and the like. It is not unusual for a single document (or, more generally, an “object”) to include content of two or more different media types or modalities. In view of this, there is interest in information retrieval systems that are capable of retrieving documents based on non-textual (visual) content. Similarity between images, for example, may be based on extracted features that are expected to have semantic significance, that is, to be discriminative of the subject matter depicted in the image. Extracted features may be based on color, shape, face recognition techniques, and the like. An information retrieval system can then use a similarity measure between a query image and stored images to retrieve a subset of the stored images which may be responsive to a user's query.
Systems have also been developed which consider one or both of text similarity and image similarity in querying multimedia collections. For example, pseudo-relevance feedback may be used. In this approach, text content of multimedia documents which have been retrieved in response to an image based query may be used to enrich the query for retrieving additional documents. Trans-media similarity methods are disclosed, for example, in S. Clinchant, J.-M. Renders, and G. Csurka, “XRCE's Participation to ImageCLEF 2007,” in Working Notes of CLEF'07 Workshop (2007); and in J. Ah-Pine, C. Cifarelli, S. Clinchant, G. Csurka, and J. Renders, “XRCE's Participation to ImageCLEF 2008,” in Working Notes of CLEF'08 Workshop (2008).
Information seeking differs from the more standard information retrieval techniques in that it does not rely solely on a query to a search engine. While information retrieval is useful for finding an answer to a specific question, such the date of birth of a particular famous person, information seeking is frequently directed more generally towards a topic, such as seeking events in the life of the famous person. Various strategies have been developed for accessing and exploring multimedia databases, in order to acquire and discover knowledge through information seeking. One strategy is browsing and navigation: the aim is to browse a large digital library in order to have a general overview of the different themes and the underlying structure using a tool that groups together similar objects and visualizes the similarity relations between them. The user can then explore these clusters, by zooming into particular areas, visiting specific documents and jumping to their neighbors. Another strategy is query-based searching: the aim is to find relevant objects with respect to a given query quickly using a tool that takes into account the user feedback to bridge the semantic gap between the user's query and the multimedia objects. In this case, visualizing the similarity relationships between the retrieved objects allow the user to have a more rapid understanding of the different topics and sub-topics.
A more general “mixed-strategy” approach occurs when the user wants to have a mix between serendipitous searching and query-based searching. This is often because it is hard for the user to formulate an unambiguous query, which is the direct translation of the user's information needs. It may also be the case that the user does not know exactly for what he or she is looking. It would thus be advantageous for the user to engage in a discovery process, where the user could incrementally specify more precisely the user's requirements depending on what the system is able to propose as responsive objects and for this to occur interactively, where the user could understand the direction currently being investigated with respect to the global picture, and where the user can go back to explore new directions, being aware of the boundaries of this discovery process.